Mobile terminal and method of displaying object related information therein

ABSTRACT

A mobile terminal and a method of displaying an object related information in the mobile terminal are disclosed, in which the method includes receiving an image via a camera module of the mobile terminal, the received image depicting objects located within a predetermined distance from the mobile terminal, obtaining at least one object related information corresponding to the received image, displaying the received image including the obtained at least one object related information, receiving a zoom-in command signal for the camera module, zooming in on the displayed image according to the received zoom-in command signal such that the zoomed-in image is displayed at a zoom position, and displaying object related information in the obtained object related information corresponding to at least one object of the zoomed-in image depicted between the zoom position and the predetermined distance.

CROSS-REFERENCE TO RELATED APPLICATIONS

Pursuant to 35 U.S.C. §119(a), this application claims the benefit ofearlier filing date and right of priority to Korean Application Nos.10-2010-0056480 and 10-2010-0056481 both filed on Jun. 15, 2010, thecontents of which are hereby incorporated by reference herein in theirentirety.

TECHNICAL FIELD

The present invention relates to a mobile terminal, and moreparticularly, to a mobile terminal and method of displaying objectrelated information therein. Although the present invention is suitablefor a wide scope of applications, it is particularly suitable fordisplaying object related information within an image input via acamera.

DISCUSSION OF THE RELATED ART

Generally, terminals can be classified into mobile/portable terminalsand stationary terminals. The mobile terminals can be classified intohandheld terminals and vehicle mount terminals again according topossibility of user's direct portability.

As functions of the terminal are diversified, the terminal isimplemented as a multimedia player provided with composite functionssuch as photographing of photos or moving pictures, playback of music ormoving picture files, game play, broadcast reception and the like forexample.

To support and increase of the terminal functions, it may be able toconsider the improvement of structural part and/or software part of theterminal.

However, in case of zooming in on a currently displayed preview image,the related art mobile terminal is unable to further receive an input ofa zoom-in command signal in a state that the zoom-in is not availabledue to a blocked building and the like. And, the related art mobileterminal is unable to provide other information in accordance with azoom-in action.

Moreover, in case of displaying object related information within apreview image, if the preview image is not clearly displayed, therelated art mobile terminal is unable to clearly recognize an objectcorresponding to the displayed object related information.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a mobile terminal andmethod of displaying object related information therein thatsubstantially obviate one or more problems due to limitations anddisadvantages of the related art.

An object of the present invention is to provide a mobile terminal andmethod of displaying object related information therein, by which adisplay of the object related information can be controlled according toa zoom-in extent of an image.

Another object of the present invention is to provide a mobile terminaland method of displaying object related information therein, by which adisplay of the object related information can be controlled inaccordance with a zoom-in action in case of relieving an input of azoom-in command signal in a zoom-in unavailable state.

Another object of the present invention is to provide a mobile terminaland method of displaying object related information therein, by which anobject included in an image can be identifiably displayed in case thatobject related information corresponding to the object included in theimage exists among at least one or more object related informationcorresponding to the image input via a camera.

A further object of the present invention is to provide a mobileterminal and method of displaying object related information therein, bywhich a specific object included in an image can be identifiablydisplayed if the image input via a camera is not clear.

Additional advantages, objects, and features of the invention will beset forth in part in the description which follows and in part willbecome apparent to those having ordinary skill in the art uponexamination of the following or may be learned from practice of theinvention. The objectives and other advantages of the invention may berealized and attained by the structure particularly pointed out in thewritten description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, amobile terminal according to the present invention includes a cameramodule configured to receive an image that depicts at least one objectlocated within a predetermined distance from the mobile terminal, acontroller configured to obtain at least one object related informationcorresponding to the received image and control the camera module tozoom in on the received image according to a zoom-in command signal, anda display module configured to display the zoomed-in image at a zoomposition and object related information in the obtained at least oneobject related information corresponding to at least one object of thezoomed-in image depicted between the zoom position and the predetermineddistance.

Preferably, the controller determines whether a specific objectcorresponding to a specific object related information among the atleast one object related information is included in the displayed image.If the specific object is included in the displayed image, thecontroller controls the display unit to identifiably display thespecific object.

In another aspect of the present invention, a method of displaying anobject related information in a mobile terminal includes the steps ofreceiving an image via a camera module of the mobile terminal, thereceived image depicting objects located within a predetermined distancefrom the mobile terminal, obtaining at least one object relatedinformation corresponding to the received image, displaying the receivedimage including the obtained at least one object related information,receiving a zoom-in command signal for the camera module, zooming in onthe displayed image according to the received zoom-in command signalsuch that the zoomed-in image is displayed at a zoom position, anddisplaying object related information in the obtained object relatedinformation corresponding to at least one object of the zoomed-in imagedepicted between the zoom position and the predetermined distance.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 is a block diagram of a mobile terminal according to oneembodiment of the present invention;

FIG. 2A is a front perspective diagram of a mobile terminal according toone embodiment of the present invention;

FIG. 2B is a rear perspective diagram of a mobile terminal according toone embodiment of the present invention;

FIG. 3 is a flowchart of a method of displaying object relatedinformation in a mobile terminal according to a first embodiment of thepresent invention;

FIGS. 4A to 4C are diagrams of screen configurations for displayingdifferent object related information on a same object in accordance witha zoom-in extent according to the present invention;

FIG. 5A and FIG. 5B are diagrams of screen configurations for enlarginga distance of providing object related information in accordance with azoom-in extent according to the present invention;

FIGS. 6A to 6D are diagrams of screen configurations for displayinginternal object information on a specific object in case of receiving azoom-in command signal in a zoom-in unavailable state according to thepresent invention;

FIGS. 7A to 7C are diagrams of screen configurations for displaying mapinformation including an object blocked by a specific object in case ofreceiving a zoom-in command signal in a zoom-in unavailable stateaccording to the present invention;

FIG. 8A and FIG. 8B are diagrams of screen configurations for displayinga virtual image including an object blocked by a specific object in caseof receiving a zoom-in command signal in a zoom-in unavailable stateaccording to the present invention;

FIG. 9A and FIG. 9B are diagrams of screen configurations for displayingspot information of an object blocked by a specific object in case ofreceiving a zoom-in command signal in a zoom-in unavailable stateaccording to the present invention;

FIG. 10A and FIG. 10B are diagrams of screen configurations fordisplaying path information on a path from a current terminal positionto a specific object in case of receiving a selection of a specificobject included in an image according to the present invention;

FIGS. 11A to 11C are diagrams of screen configurations for a case that acamera direction is changed in a zoom-in state according to the presentinvention;

FIG. 12 is a flowchart of a method of displaying object relatedinformation in a mobile terminal according to a second embodiment of thepresent invention;

FIGS. 13A to 13D are diagrams of screen configurations for receiving aninput of a identifiable display command signal for a specific objectincluded in a preview image according to the present invention;

FIGS. 14A to 14C are diagrams of screen configurations for displaying aspecific object identifiably using a similar image according to thepresent invention;

FIGS. 15A to 15C are diagrams of screen configurations for displaying aspecific object identifiably using a 3D graphic information according tothe present invention;

FIG. 16A and FIG. 16B are diagrams of screen configurations fordisplaying a specific object identifiably using an outline informationaccording to the present invention; and

FIG. 17 is a diagram of screen configuration for controlling a displayof a preview image using brightness adjustment according to the presentinvention.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawing figures which form a part hereof, and which show byway of illustration specific embodiments of the invention. It is to beunderstood by those of ordinary skill in this technological field thatother embodiments may be utilized, and structural, electrical, as wellas procedural changes may be made without departing from the scope ofthe present invention. Wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or similarparts. The suffixes ‘module’ and ‘unit’ for the elements used in thefollowing description are given or used in common by consideringfacilitation in writing this disclosure only but fail to have meaningsor roles discriminated from each other.

First of all, mobile terminals described in this disclosure can includea mobile phone, a smart phone, a laptop computer, a digital broadcastterminal, a PDA (personal digital assistants), a PMP (portablemultimedia player), a navigation system and the like.

Except a case applicable to a mobile terminal only, it is apparent tothose skilled in the art that the configurations according to anembodiment described in this disclosure is applicable to such astationary terminal as a digital TV, a desktop computer and the like.

FIG. 1 is a block diagram of a mobile terminal according to oneembodiment of the present invention.

Referring to FIG. 1, a mobile terminal 100 according to one embodimentof the present invention includes a wireless communication unit 110, anA/V (audio/video) input unit 120, a user input unit 130, a sensing unit140, an output unit 150, a memory 160, an interface unit 170, acontroller 180, a power supply unit 190 and the like. FIG. 1 shows themobile terminal 100 having various components, but it is understood thatimplementing all of the illustrated components is not a requirement.Greater or fewer components may alternatively be implemented.

In the following description, the above elements of the mobile terminal100 are explained in sequence.

First of all, the wireless communication unit 110 typically includes oneor more components which permits wireless communication between themobile terminal 100 and a wireless communication system or networkwithin which the mobile terminal 100 is located. For instance, thewireless communication unit 110 can include a broadcast receiving module111, a mobile communication module 112, a wireless internet module 113,a short-range communication module 114, a position-location module 115and the like.

The broadcast receiving module 111 receives a broadcast signal and/orbroadcast associated information from an external broadcast managingserver via a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrialchannel.

The broadcast managing server generally refers to a server whichgenerates and transmits a broadcast signal and/or broadcast associatedinformation or a server which is provided with a previously generatedbroadcast signal and/or broadcast associated information and thentransmits the provided signal or information to a terminal. Thebroadcast signal may be implemented as a TV broadcast signal, a radiobroadcast signal, and a data broadcast signal, among others. If desired,the broadcast signal may further include a broadcast signal combinedwith a TV or radio broadcast signal.

The broadcast associated information includes information associatedwith a broadcast channel, a broadcast program, a broadcast serviceprovider, etc. And, the broadcast associated information can be providedvia a mobile communication network. In this case, the broadcastassociated information can be received by the mobile communicationmodule 112.

The broadcast associated information can be implemented in variousforms. For instance, broadcast associated information may include anelectronic program guide (EPG) of digital multimedia broadcasting (DMB)and electronic service guide (ESG) of digital video broadcast-handheld(DVB-H).

The broadcast receiving module 111 may be configured to receivebroadcast signals transmitted from various types of broadcast systems.By nonlimiting example, such broadcasting systems include digitalmultimedia broadcasting-terrestrial (DMB-T), digital multimediabroadcasting-satellite (DMB-S), digital video broadcast-handheld(DVB-H), the data broadcasting system known as media forward link only(MediaFLO®) and integrated services digital broadcast-terrestrial(ISDB-T). Optionally, the broadcast receiving module 111 can beconfigured suitable for other broadcasting systems as well as theabove-explained digital broadcasting systems.

The broadcast signal and/or broadcast associated information received bythe broadcast receiving module 111 may be stored in a suitable device,such as a memory 160.

The mobile communication module 112 transmits/receives wireless signalsto/from one or more network entities (e.g., base station, externalterminal, server, etc.). Such wireless signals may represent audio,video, and data according to text/multimedia message transceivings,among others.

The wireless internet module 113 supports Internet access for the mobileterminal 100. This module may be internally or externally coupled to themobile terminal 100. In this case, the wireless Internet technology caninclude WLAN (Wireless LAN), Wi-Fi (Wireless Fidelity), Wibro (Wirelessbroadband), Wimax (World Interoperability for Microwave Access), HSDPA(High Speed Downlink Packet Access), etc.

The short-range communication module 114 facilitates relativelyshort-range communications. Suitable technologies for implementing thismodule include radio frequency identification (RFID), infrared dataassociation (IrDA), ultra-wideband (UWB), as well at the networkingtechnologies commonly referred to as Bluetooth and ZigBee, to name afew.

The position-location module 115 identifies or otherwise obtains thelocation of the mobile terminal 100. If desired, this module may beimplemented with a global positioning system (GPS) module.

Referring to FIG. 1, the audio/video (A/V) input unit 120 is configuredto provide audio or video signal input to the mobile terminal 100. Asshown, the A/V input unit 120 includes a camera 121 and a microphone122. The camera 121 receives and processes image frames of stillpictures or video, which are obtained by an image sensor in a video callmode or a photographing mode. And, the processed image frames can bedisplayed on the display module 151.

The image frames processed by the camera 121 can be stored in the memory160 or can be externally transmitted via the wireless communication unit110. Optionally, at least two cameras 121 can be provided to the mobileterminal 100 according to environment of usage.

The microphone 122 receives an external audio signal while the portabledevice is in a particular mode, such as phone call mode, recording modeand voice recognition. This audio signal is processed and converted intoelectric audio data. The processed audio data is transformed into aformat transmittable to a mobile communication base station via themobile communication module 112 in case of a call mode. The microphone122 typically includes assorted noise removing algorithms to removenoise generated in the course of receiving the external audio signal.

The user input unit 130 generates input data responsive to usermanipulation of an associated input device or devices. Examples of suchdevices include a keypad, a dome switch, a touchpad (e.g., staticpressure/capacitance), a jog wheel, a jog switch, etc.

The sensing unit 140 provides sensing signals for controlling operationsof the mobile terminal 100 using status measurements of various aspectsof the mobile terminal. For instance, the sensing unit 140 may detect anopen/close status of the mobile terminal 100, relative positioning ofcomponents (e.g., a display and keypad) of the mobile terminal 100, achange of position of the mobile terminal 100 or a component of themobile terminal 100, a presence or absence of user contact with themobile terminal 100, orientation or acceleration/deceleration of themobile terminal 100. As an example, consider the mobile terminal 100being configured as a slide-type mobile terminal. In this configuration,the sensing unit 140 may sense whether a sliding portion of the mobileterminal is open or closed. Other examples include the sensing unit 140sensing the presence or absence of power provided by the power supply190, the presence or absence of a coupling or other connection betweenthe interface unit 170 and an external device. And, the sensing unit 140can include a proximity sensor 141.

The output unit 150 generates outputs relevant to the senses of sight,hearing, touch and the like. And, the output unit 150 includes thedisplay module 151, an audio output module 152, an alarm module 153, ahaptic module 154, a projector module 155 and the like.

The display module 151 is typically implemented to visually display(output) information associated with the mobile terminal 100. Forinstance, if the mobile terminal is operating in a phone call mode, thedisplay will generally provide a user interface (UI) or graphical userinterface (GUI) which includes information associated with placing,conducting, and terminating a phone call. As another example, if themobile terminal 100 is in a video call mode or a photographing mode, thedisplay module 151 may additionally or alternatively display imageswhich are associated with these modes, the UI or the GUI.

The display module 151 may be implemented using known displaytechnologies including, for example, a liquid crystal display (LCD), athin film transistor-liquid crystal display (TFT-LCD), an organiclight-emitting diode display (OLED), a flexible display and athree-dimensional (3D) display. The mobile terminal 100 may include oneor more of such displays.

Some of the above displays can be implemented in a transparent oroptical transmittive type, which can be named a transparent display. Asa representative example for the transparent display, there is TOLED(transparent OLED) or the like. A rear configuration of the displaymodule 151 can be implemented in the optical transmittive type as well.In this configuration, a user is able to see an object in rear of aterminal body via the area occupied by the display module 151 of theterminal body.

At least two display units 151 can be provided to the mobile terminal100 in accordance with the implemented configuration of the mobileterminal 100. For instance, a plurality of display units can be arrangedon a single face of the mobile terminal 100 in a manner of being spacedapart from each other or being built in one body. Alternatively, aplurality of display units can be arranged on different faces of themobile terminal 100.

In case that the display module 151 and a sensor for detecting a touchaction (hereinafter called ‘touch sensor’) configures a mutual layerstructure (hereinafter called ‘touchscreen’), it is able to use thedisplay module 151 as an input device as well as an output device. Inthis case, the touch sensor can be configured as a touch film, a touchsheet, a touchpad or the like.

The touch sensor can be configured to convert a pressure applied to aspecific portion of the display module 151 or a variation of acapacitance generated from a specific portion of the display module 151to an electric input signal. Moreover, it is able to configure the touchsensor to detect a pressure of a touch as well as a touched position orsize.

If a touch input is made to the touch sensor, signal(s) corresponding tothe touch is transferred to a touch controller. The touch controllerprocesses the signal(s) and then transfers the processed signal(s) tothe controller 180. Therefore, the controller 180 is able to knowwhether a prescribed portion of the display module 151 is touched.

Referring to FIG. 1, a proximity sensor 141 can be provided to aninternal area of the mobile terminal 100 enclosed by the touchscreen oraround the touchscreen. The proximity sensor 141 is the sensor thatdetects a presence or non-presence of an object approaching a prescribeddetecting surface or an object existing around the proximity sensor 141using an electromagnetic field strength or infrared ray withoutmechanical contact. Hence, the proximity sensor 141 has durabilitylonger than that of a contact type sensor and also has utility widerthan that of the contact type sensor.

The proximity sensor 141 can include one of a transmittive photoelectricsensor, a direct reflective photoelectric sensor, a mirror reflectivephotoelectric sensor, a radio frequency oscillation proximity sensor141, an electrostatic capacity proximity sensor 141, a magneticproximity sensor 141, an infrared proximity sensor 141 and the like. Incase that the touchscreen includes the electrostatic capacity proximitysensor 141, it is configured to detect the proximity of a pointer usinga variation of electric field according to the proximity of the pointer.In this case, the touchscreen (touch sensor) can be classified as theproximity sensor 141.

In the following description, for clarity, an action that a pointerapproaches without contacting with the touchscreen to be recognized aslocated on the touchscreen is named ‘proximity touch’. And, an actionthat a pointer actually touches the touchscreen is named ‘contacttouch’. The meaning of the position on the touchscreen proximity-touchedby the pointer means the position of the pointer which verticallyopposes the touchscreen when the pointer performs the proximity touch.

The proximity sensor 141 detects a proximity touch and a proximity touchpattern (e.g., a proximity touch distance, a proximity touch duration, aproximity touch position, a proximity touch shift state, etc.). And,information corresponding to the detected proximity touch action and thedetected proximity touch pattern can be output to the touchscreen.

The audio output module 152 functions in various modes including acall-receiving mode, a call-placing mode, a recording mode, a voicerecognition mode, a broadcast reception mode and the like to outputaudio data which is received from the wireless communication unit 110 oris stored in the memory 160. During operation, the audio output module152 outputs audio relating to a particular function (e.g., callreceived, message received, etc.). The audio output module 152 is oftenimplemented using one or more speakers, buzzers, other audio producingdevices, and combinations thereof.

The alarm module 153 is output a signal for announcing the occurrence ofa particular event associated with the mobile terminal 100. Typicalevents include a call received event, a message received event and atouch input received event. The alarm module 153 is able to output asignal for announcing the event occurrence by way of vibration as wellas video or audio signal. The video or audio signal can be output viathe display module 151 or the audio output unit 152. Hence, the displaymodule 151 or the audio output module 152 can be regarded as a part ofthe alarm module 153.

The haptic module 154 generates various tactile effects that can besensed by a user. Vibration is a representative one of the tactileeffects generated by the haptic module 154. Strength and pattern of thevibration generated by the haptic module 154 are controllable. Forinstance, different vibrations can be output in a manner of beingsynthesized together or can be output in sequence.

The haptic module 154 is able to generate various tactile effects aswell as the vibration. For instance, the haptic module 154 generates theeffect attributed to the arrangement of pins vertically moving against acontact skin surface, the effect attributed to the injection/suctionpower of air though an injection/suction hole, the effect attributed tothe skim over a skin surface, the effect attributed to the contact withelectrode, the effect attributed to the electrostatic force, the effectattributed to the representation of hold/cold sense using an endothermicor exothermic device and the like.

The haptic module 154 can be implemented to enable a user to sense thetactile effect through a muscle sense of finger, arm or the like as wellas to transfer the tactile effect through a direct contact. Optionally,at least two haptic modules 154 can be provided to the mobile terminal100 in accordance with the corresponding configuration type of themobile terminal 100.

The projector module 155 is the element for performing an imageprojector function using the mobile terminal 100. And, the projectormodule 155 is able to display an image, which is identical to orpartially different at least from the image displayed on the displaymodule 151, on an external screen or wall according to a control signalof the controller 180.

In particular, the projector module 155 can include a light source (notshown in the drawing) generating light (e.g., laser) for projecting animage externally, an image producing means (not shown in the drawing)for producing an image to output externally using the light generatedfrom the light source, and a lens (not shown in the drawing) forenlarging to output the image externally in a predetermined focusdistance. And, the projector module 155 can further include a device(not shown in the drawing) for adjusting an image projected direction bymechanically moving the lens or the whole module.

The projector module 155 can be classified into a CRT (cathode ray tube)module, an LCD (liquid crystal display) module, a DLP (digital lightprocessing) module or the like according to a device type of a displaymeans. In particular, the DLP module is operated by the mechanism ofenabling the light generated from the light source to reflect on a DMD(digital micro-mirror device) chip and can be advantageous for thedownsizing of the projector module 151.

Preferably, the projector module 155 can be provided in a lengthdirection of a lateral, front or backside direction of the mobileterminal 100. And, it is understood that the projector module 155 can beprovided to any portion of the mobile terminal 100 according to thenecessity thereof.

The memory unit 160 is generally used to store various types of data tosupport the processing, control, and storage requirements of the mobileterminal 100. Examples of such data include program instructions forapplications operating on the mobile terminal 100, contact data,phonebook data, messages, audio, still pictures, moving pictures, etc.And, a recent use history or a cumulative use frequency of each data(e.g., use frequency for each phonebook, each message or eachmultimedia) can be stored in the memory unit 160. Moreover, data forvarious patterns of vibration and/or sound output in case of a touchinput to the touchscreen can be stored in the memory unit 160.

The memory 160 may be implemented using any type or combination ofsuitable volatile and non-volatile memory or storage devices includinghard disk, random access memory (RAM), static random access memory(SRAM), electrically erasable programmable read-only memory (EEPROM),erasable programmable read-only memory (EPROM), programmable read-onlymemory (PROM), read-only memory (ROM), magnetic memory, flash memory,magnetic or optical disk, multimedia card micro type memory, card-typememory (e.g., SD memory, XD memory, etc.), or other similar memory ordata storage device. And, the mobile terminal 100 is able to operate inassociation with a web storage for performing a storage function of thememory 160 on Internet.

The interface unit 170 is often implemented to couple the mobileterminal 100 with external devices. The interface unit 170 receives datafrom the external devices or is supplied with the power and thentransfers the data or power to the respective elements of the mobileterminal 100 or enables data within the mobile terminal 100 to betransferred to the external devices. The interface unit 170 may beconfigured using a wired/wireless headset port, an external chargerport, a wired/wireless data port, a memory card port, a port forcoupling to a device having an identity module, audio input/outputports, video input/output ports, an earphone port and/or the like.

The identity module is the chip for storing various kinds of informationfor authenticating a use authority of the mobile terminal 100 and caninclude User Identity Module (UIM), Subscriber Identity Module (SIM),Universal Subscriber Identity Module (USIM) and/or the like. A devicehaving the identity module (hereinafter called ‘identity device’) can bemanufactured as a smart card. Therefore, the identity device isconnectible to the mobile terminal 100 via the corresponding port.

When the mobile terminal 110 is connected to an external cradle, theinterface unit 170 becomes a passage for supplying the mobile terminal100 with a power from the cradle or a passage for delivering variouscommand signals input from the cradle by a user to the mobile terminal100. Each of the various command signals input from the cradle or thepower can operate as a signal enabling the mobile terminal 100 torecognize that it is correctly loaded in the cradle.

The controller 180 typically controls the overall operations of themobile terminal 100. For example, the controller 180 performs thecontrol and processing associated with voice calls, data communications,video calls, etc. The controller 180 may include a multimedia module 181that provides multimedia playback. The multimedia module 181 may beconfigured as part of the controller 180, or implemented as a separatecomponent.

Moreover, the controller 180 is able to perform a pattern recognizingprocess for recognizing a writing input and a picture drawing inputcarried out on the touchscreen as characters or images, respectively.

The power supply unit 190 provides power required by the variouscomponents for the mobile terminal 100. The power may be internal power,external power, or combinations thereof.

Various embodiments described herein may be implemented in acomputer-readable medium using, for example, computer software,hardware, or some combination thereof. For a hardware implementation,the embodiments described herein may be implemented within one or moreapplication specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), processors, controllers, micro-controllers, microprocessors,other electronic units designed to perform the functions describedherein, or a selective combination thereof. Such embodiments may also beimplemented by the controller 180.

For a software implementation, the embodiments described herein may beimplemented with separate software modules, such as procedures andfunctions, each of which perform one or more of the functions andoperations described herein. The software codes can be implemented witha software application written in any suitable programming language andmay be stored in memory such as the memory 160, and executed by acontroller or processor, such as the controller 180.

FIG. 2A is a front perspective diagram of a mobile terminal according toone embodiment of the present invention.

The mobile terminal 100 shown in the drawing has a bar type terminalbody. Yet, the mobile terminal 100 may be implemented in a variety ofdifferent configurations. Examples of such configurations includefolder-type, slide-type, rotational-type, swing-type and combinationsthereof. For clarity, further disclosure will primarily relate to abar-type mobile terminal 100. However such teachings apply equally toother types of mobile terminals.

Referring to FIG. 2A, the mobile terminal 100 includes a case (casing,housing, cover, etc.) configuring an exterior thereof. In the presentembodiment, the case can be divided into a front case 101 and a rearcase 102. Various electric/electronic parts are loaded in a spaceprovided between the front and rear cases 101 and 102. Optionally, atleast one middle case can be further provided between the front and rearcases 101 and 102 in addition.

The cases 101 and 102 are formed by injection molding of synthetic resinor can be formed of metal substance such as stainless steel (STS),titanium (Ti) or the like for example.

A display module 151, an audio output unit 152, a camera 121, user inputunits 131 and 132, a microphone 122, an interface 180 and the like canbe provided to the terminal body, and more particularly, to the frontcase 101.

The display module 151 occupies most of a main face of the front case101. The audio output unit 151 and the camera 121 are provided to anarea adjacent to one of both end portions of the display module 151,while the user input unit 131 and the microphone 122 are provided toanother area adjacent to the other end portion of the display module151. The user input unit 132 and the interface 170 can be provided tolateral sides of the front and rear cases 101 and 102.

The input unit 130 is manipulated to receive a command for controllingan operation of the terminal 100. And, the input unit 130 is able toinclude a plurality of manipulating units 131 and 132. The manipulatingunits 131 and 132 can be named a manipulating portion and may adopt anymechanism of a tactile manner that enables a user to perform amanipulation action by experiencing a tactile feeling.

Content input by the first or second manipulating unit 131 or 132 can bediversely set. For instance, such a command as start, end, scroll andthe like is input to the first manipulating unit 131. And, a command fora volume adjustment of sound output from the audio output unit 152, acommand for a switching to a touch recognizing mode of the displaymodule 151 or the like can be input to the second manipulating unit 132.

FIG. 2B is a perspective diagram of a backside of the terminal shown inFIG. 2A.

Referring to FIG. 2B, a camera 121′ can be additionally provided to abackside of the terminal body, and more particularly, to the rear case102. The camera 121 has a photographing direction that is substantiallyopposite to that of the former camera 121 shown in FIG. 2A and may havepixels differing from those of the firmer camera 121.

Preferably, for instance, the former camera 121 has low pixels enough tocapture and transmit a picture of user's face for a video call, whilethe latter camera 121′ has high pixels for capturing a general subjectfor photography without transmitting the captured subject. And, each ofthe cameras 121 and 121′ can be installed at the terminal body to berotated or popped up.

A flash 123 and a mirror 124 are additionally provided adjacent to thecamera 121′. The flash 123 projects light toward a subject in case ofphotographing the subject using the camera 121′. In case that a userattempts to take a picture of the user (self-photography) using thecamera 121′, the mirror 124 enables the user to view user's facereflected by the mirror 124.

An additional audio output unit 152′ can be provided to the backside ofthe terminal body. The additional audio output unit 152′ is able toimplement a stereo function together with the former audio output unit152 shown in FIG. 2A and may be used for implementation of aspeakerphone mode in talking over the terminal.

A broadcast signal receiving antenna 124 can be additionally provided tothe lateral side of the terminal body as well as an antenna forcommunication or the like. The antenna 124 constructing a portion of thebroadcast receiving module 111 shown in FIG. 1 can be retractablyprovided to the terminal body.

A power supply unit 190 for supplying a power to the terminal 100 isprovided to the terminal body. And, the power supply unit 190 can beconfigured to be built within the terminal body. Alternatively, thepower supply unit 190 can be configured to be detachably connected tothe terminal body.

A touchpad 135 for detecting a touch can be additionally provided to therear case 102. The touchpad 135 can be configured in a lighttransmittive type like the display module 151. In this case, if thedisplay module 151 is configured to output visual information from itsboth faces, it is able to recognize the visual information via thetouchpad 135 as well. The information output from both of the faces canbe entirely controlled by the touchpad 135. Alternatively, a display isfurther provided to the touchpad 135 so that a touchscreen can beprovided to the rear case 102 as well.

The touchpad 135 is activated by interconnecting with the display module151 of the front case 101. The touchpad 135 can be provided in rear ofthe display module 151 in parallel. The touchpad 135 can have a sizeequal to or smaller than that of the display module 151.

First of all, a mobile terminal mentioned in the following descriptioncan include at least one of the components shown in FIG. 1. Moreover, amobile terminal can include a camera for receiving an input of an imageexternally and photographing the input image.

In the following description, a method of displaying object relatedinformation in a mobile terminal according to the present invention isexplained with reference to the accompanying drawings.

First of all a first embodiment of the present invention is explainedwith reference to FIGS. 3 to 11C.

FIG. 3 is a flowchart of a method of displaying object relatedinformation in a mobile terminal according to a first embodiment of thepresent invention.

Referring to FIG. 3, the mobile terminal 100 receives an input of animage externally via the camera 121 [S310]. In this case, the inputimage is a preview image and can be displayed via the display module151.

In case that a camera photograph mode or an augmented reality drive modeis entered, the mobile terminal 100 drives the camera 121 and is thenable to receive an external image using the driven camera 121. In thiscase, the camera photograph mode or the augmented reality drive mode canbe entered if a corresponding menu item or a corresponding key (or acorresponding key region) is selected. Moreover, while the previewimage, which is input in the camera photograph mode using the camera121, is displayed, the augmented reality mode can be entered as well.

In this case, the augmented reality belongs to one field of virtualreality and is a computer graphic scheme of synthesizing a realenvironment with a virtual object to look as if existing in an originalenvironment. Therefore, the mobile terminal 100 synthesizes a virtualobject or image onto an image representing a real environment input viathe camera 121 in the augmented reality drive mode and is then able todisplay the synthesized image.

And, it is able to zoom in or out on the input image according to a usersetting.

Moreover, the mobile terminal 100 is able to obtain current positioninformation of the mobile terminal 100 (hereinafter named currentterminal position information) using the position location module 115under the control of the controller 180. This is to obtain objectrelated information with reference to the current terminal positioninformation. And, the object related information shall be describedlater in this disclosure.

The mobile terminal 100 is able to obtain the current terminal positioninformation from a satellite by GPS (global positioning system). Forinstance, position information can include such information on a point,at which the mobile terminal 100 is currently located, as longitude,latitude, altitude, coordinates and the like.

Moreover, the position information of the mobile terminal 100 canfurther include a direction faced by the camera 121 of the mobileterminal 100 at the point at which the mobile terminal 100 is currentlylocated. In this case, the mobile terminal 100 includes a motiondetecting sensor (not shown in the drawing) and is able to detect thedirection faced by the camera 121 using the motion detecting sensor. Forinstance, the direction faced by the camera 121 can be represented as atleast one of east, west, south and north.

The mobile terminal 100 obtains at least one object related informationcorresponding to the image input in the inputting step S310 under thecontrol of the controller 180 [S320].

In particular, in the obtaining step S320, the controller 180 is able toobtain object related information on each of at least one or moreobjects located within a predetermined distance from the mobile terminal100 with reference to the obtained current terminal positioninformation. In this case, the at least one object corresponding to theobtained object related information is included in the input image orcan be located within a predetermined distance from the current terminalposition in the direction faced by the camera 121.

In this case, the object can include such a thing having addressinformation (or position information) of its own as a building, a shop,a shopping center and the like. And, the object related information isthe information related to the object and can include at least one of aname of the object (e.g., POI (point of interest)), a representativeicon of the object, position information of the object, a phone numberof the object and the like. Besides, object related information canexist on some of all objects located within a predetermined distancefrom the mobile terminal 100 only.

In more particular, the controller 180 is able to recognize a position,at which the mobile terminal 100 is currently located, using thecoordinates information in the obtained terminal position informationand is also able to recognize a side in front of the mobile terminal 100at the currently located position using the direction faced by thecamera 121 in the obtained current terminal position information.Therefore, the controller 180 is able to obtain object relatedinformation of each of a plurality of the objects located within apredetermined distance in front of the mobile terminal 100.

In the following description, the object related information obtainingstep S320 is explained in detail.

First of all, the mobile terminal 100 enables object related informationper object to be stored in the memory 160. The mobile terminal 100 isthen able to search each object related information of a plurality ofthe objects from the per-object object related information previouslystored in the memory 160 under the control of the controller 180. Inthis case, the position information of the object having thecorresponding object related information can be stored in the memory 160together with the corresponding object related information.

In particular, the controller 180 is able to search the memory 160 for aplurality of the objects having their position information belonging toa range of a predetermined distance in front of the mobile terminal 100and the object related information designated to each of a plurality ofthe objects. Therefore, if the per-object object related informationstored in the memory 160 is found, the controller 180 is able to obtainthe object related information of each of a plurality of the objectswithin the predetermined distance in front of the mobile terminal 100.

Moreover, the per-object object related information and the positioninformation previously stored in the memory 160 are received from anexternal server or can be stored as default in the course ofmanufacturing the corresponding terminal. Besides, the per-object objectrelated information and the position information previously stored inthe memory 160 can be updated periodically or at a random timing point.

Subsequently, the mobile terminal 100 is able to receive object relatedinformation corresponding to the input image from an external server viathe wireless communication unit 110.

In particular, the mobile terminal 100 transmits a signal for requestingeach object related information of a plurality of the objects locatedwithin a predetermined distance in front of the mobile terminal 100 toan external server and is then able to receive each object relatedinformation of a plurality of the objects located within a predetermineddistance in front of the mobile terminal 100.

In this case, the request signal includes each position information of aplurality of the objects [former case] or a predetermined distance fordisplaying the position information and object related information ofthe mobile terminal 100 [latter case].

In the former case, the mobile terminal 100 is able to receive theobject related information corresponding to the position informationincluded in the request signal from the external server. In the lattercase, the mobile terminal 100 is able to receive each object relatedinformation of a plurality of the objects located within thepredetermined distance in front from the current location of the mobileterminal 100 from the external server.

Under the control of the controller 180, the mobile terminal 100displays the image input in the inputting step S310 and is also able todisplay the object related information of a plurality of the objects,which were obtained in the obtaining step S320, within the input image[S330]. In this case, the displaying step S330 can be performed usingthe display module 151.

In doing so, the object related information can displayed within theinput image by being linked to an object image corresponding to theobject. For instance, if the object related information indicates ‘LGElectronics’ and the corresponding object is an LG Electronics building,the object related information ‘LG Electronics’ can be displayed withinthe input image by being linked to an image of the LG Electronicsbuilding.

Even if an object image corresponding to the object related informationdoes not exist within the input image, the object related informationcan be displayed at a predicted point where the corresponding object mayexist.

Meanwhile, the mobile terminal 100 receives an input of a zoom-incommand signal for the camera 121 via the user input unit 130 [S340]. Incase of receiving the input of the zoom-in command signal, the mobileterminal 100 zooms in on the image input via the camera 121 under thecontrol of the controller 180 [S350].

In this case, the zoom-in command signal can include a signal having aneffect on the camera 121 to receive an input of an image at a positioncloser to a subject than a real position of the camera 121 in a mannerof adjusting a focal distance by fixing the position of the camera 121.

For instance, in case of receiving an input of the zoom-in commandsignal, although the position of the camera 121 is fixed, the camera 121can have an effect of receiving an input of an image of a same subjectby going forward in a predetermined distance farther than a currentposition. Hence, the image input via the camera 121 can be displayed ina manner of becoming larger than an image before the input of thezoom-in command signal. Alternatively, in case of receiving an input ofa zoom-in command signal, since an image is input by going forward in apredetermined distance farther than a current position, a subjectexisting between the current position and a forward position (virtualposition) may not be displayed.

The inputting step S340 can be performed in one of a case that a menuitem, a key or a key region corresponding to a generation of a zoom-incommand signal is selected by a user, a case that a terminal motion(e.g., backward/forward incline, a predetermined count of shakings,etc.) corresponding to a zoom-in command signal is detected, and a casethat a touch action corresponding to a zoom-in command signal is inputby a user.

In the inputting step S340, while the obtained object relatedinformation is displayed within the image input via the camera 121, thezoom-in command signal for the input image can be input.

The mobile terminal 100 controls a display of the object relatedinformation displayed within the input image to correspond to a zoom-inextent of the image under the control of the controller 180 [S360].

In the controlling step S360, the mobile terminal 100 is able to set theobject related information to differ for the same object to correspondto the zoom-in extent of the image under the control of the controller180.

This is explained in detail with reference to FIGS. 4A to 5B as follows.For clarity and convenience, the following description is made bylimiting the input image to a preview image for example. And, assumethat the object related information corresponding to the input imageincludes information on an object located within 500 meters in frontfrom a current terminal position.

FIGS. 4A to 4C are diagrams of screen configurations for displayingdifferent object related information on a same object in accordance witha zoom-in extent according to the present invention.

Referring to FIG. 4A, the mobile terminal 100 displays a preview image,which is input via the camera 121, on a screen. In doing so, the mobileterminal 100 is able to display object related information (e.g., names)401 to 405 on a plurality of object images included in the previewimage, respectively, in a manner that each of the object relatedinformation is linked to the corresponding object image.

For instance, the object related information displayed in not-zoom-instate can be assumed as a building name (or a company or shop name).

Meanwhile, in case that a direction faced by the camera 121 is changedaccording to a motion of the mobile terminal 100 or camera 121, themobile terminal 100 receives an input of a preview image correspondingto the changed direction and then displays the input preview image. And,the mobile terminal 100 is also able to display object images of objectslocated within a predetermined front distance with reference to thechanged direction in a manner of updating object related information ofthe objects [not shown in the drawing].

In case of receiving an input of a first zoom-in command signal (e.g., azoom-in distance is assumed as 100M) in FIG. 4A, the mobile terminal 100receives an input of an image not at a current terminal position but ata 100M-forward position and is then able to display object relatedinformation corresponding to the image input at the 100M-ahead position[FIG. 4B].

Referring to FIG. 4B, the object related information corresponding tothe input image can include the object related information belonging toa range between 100M and 500M in front instead of including the objectrelated information belonging to the front 100M from the currentterminal position.

For instance, in case that a zoom-in extent is 100M, the object relatedinformation can include a corresponding phone number and a correspondinghomepage address as well as a building name.

In case of receiving an input of a second zoom-in command signal in FIG.4A or FIG. 4B (e.g., a zoom-in distance is assumed as 200M) in FIG. 4A,the mobile terminal 100 receives an input of an image not at a currentterminal position but at a 200M-ahead position and is then able todisplay object related information corresponding to the image input atthe 200M-ahead position [FIG. 4C].

Referring to FIG. 4C, the object related information corresponding tothe input image can include the object related information belonging toa range between 200M and 500M in front instead of including the objectrelated information belonging to the front 200M from the currentterminal position.

For instance, in case that a zoom-in extent is 200M, if an object is ‘ΔΔmovie theater’, the object related information can include acorresponding phone number, an associated discount card information, anew release trail information, a reservation related site informationand the like as well as a building name.

In particular, referring to FIGS. 4A to 4C, if the zoom-in extentincrease, it can be observed that further detailed or more variousobject related information on an object included in an image can bedisplayed.

Moreover, referring to FIGS. 4A to 4C, it can be observed that anindicator (or icon) 410, which indicates that it is currently in azoom-in state, is displayed on a prescribed region of a screen.

FIG. 5A and FIG. 5B are diagrams of screen configurations for enlarginga distance of providing object related information in accordance with azoom-in extent according to the present invention.

In case of receiving an input of a first zoom-in command signal (e.g., azoom-in distance is assumed as 100M) in FIG. 4A, the mobile terminal 100receives an input of an image not at a current terminal position but ata 100M-ahead position and is then able to display object relatedinformation belonging to a range between 100M and 600M [FIG. 5A].

Referring to FIG. 5A, in case of receiving the input of the image at the100M-ahead position, in order to provide object related informationcorresponding to total 500M, it is able to display the object relatedinformation belonging to a range between 100M and 600M. In this case,the range between 500M and 600M can be named an additional 100M range.

In particular, referring to FIG. 5A (a), object related information 501and 502, each of which belongs to a range between 100M and 600M, aredisplayed at points, at which the corresponding objects may bepredictably located, respectively. Referring to FIG. 5A (b), objectrelated information (e.g., a building name, a distance from a currentterminal position, etc.) belonging to the additional 100M range can bedisplayed on a separate region 510 of the screen.

In case of receiving an input of a second zoom-in command signal in FIG.4A or FIG. 4B (e.g., a zoom-in distance is assumed as 200M) in FIG. 4A,the mobile terminal 100 receives an input of an image not at a currentterminal position but at a 200M-ahead position and is then able todisplay object related information corresponding to a range between 200Mand 700M [FIG. 5B].

Referring to FIG. 5B, in case of receiving the input of the image at the200M-ahead position, in order to provide object related informationcorresponding to total 500M, it is able to display the object relatedinformation belonging to a range between 200M and 700M. In this case,the range between 500M and 700M can be named an additional 200M range.

In particular, referring to FIG. 5B (a), object related information 501and 504 (e.g., sing-sing mart building, happy coffee shop building, ΔΔamusement park, oo hospital, etc.), each of which belongs to theadditional 200M range, are displayed at points, at which thecorresponding objects may be predictably located, respectively.Referring to FIG. 5B (b), object related information (e.g., a buildingname, a distance from a current terminal position, etc.) belonging tothe additional 200M range can be displayed on a separate region 520 ofthe screen.

Meanwhile, the object related information corresponding to theadditional range can be displayed together with a virtual preview imagecorresponding to the additional range [not shown in the drawing]. Forinstance, if there is an additional range, a real preview image and avirtual preview image (object related information included) aredisplayed in a manner of being combined with each other or a virtualpreview image is separately displayed.

Besides, the acquisition of the object related information correspondingto the additional range shall refer to the foregoing descriptionrelating to the acquisition of the object related information (i.e., thedescription of the step S320).

Referring now to FIG. 3, in the controlling step S360, in case ofreceiving the zoom-in command signal in the zoom-in unavailable state,the mobile terminal 100 is able to control to internal objectinformation on a specific object included in a currently input image tobe displayed as the object related information under the control of thecontroller 180.

When the camera 121 is facing a specific object or the camera 121 facingthe specific object is unable to further zoom in on the specific object(this state shall be named a zoom-in unavailable state hereinafter), ifthe mobile terminal 100 further receives an input of a zoom-in commandsignal, it is able to display the internal object information on thespecific object.

For instance, the internal object information can include shopinformation of a building corresponding to an object, per-floor layoutinformation, per-shop detailed information (e.g., phone number, discountcard, interest-free card, etc.), and the like.

The mobile terminal 100 enables internal object information per object,which includes internal object information on a specific object, to bestored in the memory 160 or is able to receive internal objectinformation on a specific object via the wireless communication unit110.

In particular, the mobile terminal 100 searches the per-object internalobject information stored in the memory 160 for the internal objectinformation on a specific object under the control of the controller180. Alternatively, the mobile terminal 100 transmits a request signalof internal object information on a specific object to an externalserver and is then able to receive the internal object information onthe specific object from the external server.

In this case, the per-object internal object information stored in thememory 160 can be stored together with position information of thecorresponding object. And, the position information of the specificobject can be included the request signal for the internal objectinformation.

While it is actually unable to further zoom in on an image, in case ofreceiving the zoom-in command signal, the mobile terminal 100 is able toperform a display action in accordance with the zoom-in in associationwith a specific object.

For example, it is able to display internal object information on aspecific object. This is explained in detail with reference to FIGS. 6Ato 6D. For clarity and convenience of the following description, assumethat a specific object is the ΔΔ movie theater. And, assume that firstto third zoom-in command signals mentioned in the following descriptionare the zoom-in command signals input in a zoom-in unavailable state andhave zoom-in extents increasing in order, respectively.

FIGS. 6A to 6D are diagrams of screen configurations for displayinginternal object information on a specific object in case of receiving azoom-in command signal in a zoom-in unavailable state according to thepresent invention.

Referring to FIG. 6A, the ΔΔ movie theater and object relatedinformation (building name) 601 of the ΔΔ movie theater are displayedwithin an image input via the camera 121 in a state that zoom-in is notavailable further.

In case that the first zoom-in command signal is input in FIG. 6A, themobile terminal 100 is able to display per-floor shop information of theΔΔ movie theater [FIG. 6B].

While a fourth floor (4F) 620 is selected, in case that the secondzoom-in command signal is input in FIG. 6B, the mobile terminal 100 isable to display a layout of the shops located on the fourth floor of theΔΔ movie theater [FIG. 6C].

While a third screen located on the fourth floor is selected in FIG. 6C,if the second zoom-in command signal is input, the mobile terminal 100is able to display a detailed layout of the third screen [FIG. 6D].

Referring now to FIG. 3, in the controlling step S360, in case ofreceiving the zoom-in command signal in the zoom-in unavailable state,the mobile terminal 100 is able to control information on at least oneobject, which is blocked by the specific object included in thecurrently input image, to be displayed as the object related informationunder the control of the controller 180.

In particular, in case of receiving an input of the zoom-in commandsignal in a state that it is actually unable to further zoom in on theimage, the mobile terminal 100 enables the display operation inaccordance with the zoom-in to be performed in association with thespecific object.

For instance, the mobile terminal 100 is able display the information onthe object blocked by the specific object. This is explained in detailwith reference to FIGS. 7A to 9B as follows.

In the following description, a specific object can include an objectcurrently faced by the camera 121 or a unique object existing in thedirection faced by the camera 121. Moreover, information on at least oneobject blocked by the specific object can include at least one of mapinformation (hereinafter named virtual map information) including theblocked at least one object, a virtual image including the blocked atleast one object and point information (hereinafter named virtual pointinformation) on each of the blocked at least one object.

For instance, the virtual map information, the virtual image or thevirtual point information is stored in the memory 160 or can be receivedfrom the external server via the wireless communication unit 110.

FIGS. 7A to 7C are diagrams of screen configurations for displaying mapinformation including an object blocked by a specific object in case ofreceiving a zoom-in command signal in a zoom-in unavailable stateaccording to the present invention.

Referring to FIG. 7A, in case of receiving a zoom-in command signal in azoom-in unavailable state, the mobile terminal 100 is able to perform avirtual zoom-in operation on a specific object included in a currentimage. In doing so, the mobile terminal 100 is able to output a phrase710 indicating that the virtual zoom-in operation is in progress.

In this case, the virtual zoom-in operation on the specific object is azoom-in operation of penetrating the specific object and is able toinclude an operation for being provided with an object blocked by thespecific object and object related information on the blocked object.And, the specific object can mean a unique object faced by the camera121 currently.

Regarding FIG. 7A, the virtual zoom-in operation can be performed in thestate that the camera 121 is facing the specific object or in a statethat internal object information on the specific object is beingdisplayed [cf. FIGS. 6A to 6D]. In particular, regarding FIGS. 6A to 6D,the virtual zoom-in operation can be performed in case that a zoom-incommand signal having a zoom-in extent greater than that of the secondzoom-in command signal is input.

Referring to FIG. 7B and FIG. 7C, as the virtual zoom-in operation isperformed, the mobile terminal 100 is able to display map informationincluding the object blocked by the specific object.

In particular, in FIG. 7B and FIG. 7C, a first indicator 701 indicates acurrent terminal position and a second indicator 702 indicates a virtualterminal position according to the virtual zoom-in operation. Moreover,it can be observed that the second indicator 702 goes forward farther ifthe zoom-in extent increases further.

In FIG. 7B and FIG. 7C, an indicator 720 indicating that the virtualzoom-in operation is in progress is displayed on a prescribed region ofthe screen.

FIG. 8A and FIG. 8B are diagrams of screen configurations for displayinga virtual image including an object blocked by a specific object in caseof receiving a zoom-in command signal in a zoom-in unavailable stateaccording to the present invention.

Referring to FIG. 8A and FIG. 8B, as the virtual zoom-in operation isperformed in FIG. 7A, the mobile terminal 100 is able to display avirtual image including an object blocked by a specific object.

For instance, the virtual image displayed in FIG. 8A and FIG. 8B caninclude object related information 801 to 804 on objects located in thevirtual image. Moreover, if a zoom-in extent increases, the mobileterminal 100 is able to display the virtual image in a manner of movinga virtual terminal position forward in proportion to the zoom-in extent.

In FIG. 8A and FIG. 8B, an indicator 810 indicating that the virtualzoom-in operation is in progress is displayed on a prescribed region ofthe screen.

FIG. 9A and FIG. 9B are diagrams of screen configurations for displayingspot information of an object blocked by a specific object in case ofreceiving a zoom-in command signal in a zoom-in unavailable stateaccording to the present invention.

Referring to FIG. 9A and FIG. 9B, as the virtual zoom-in operation isperformed in FIG. 7A, the mobile terminal 100 is able to display pointinformation (hereinafter named an additional object point information)of an object blocked by a specific object.

For instance, the mobile terminal is able to display an additionalobject point information 920 on a separate region of the screen bymaintaining an image display including the specific object.

In FIG. 9A and FIG. 9B, an indicator 910 indicating that the virtualzoom-in operation is in progress is displayed on a prescribed region ofthe screen.

According to the present invention, the mobile terminal 100 furtherincludes a user input unit 130 receiving an input of a selection actionfor a random object included in an image input via the camera 121 (animage in a zoom-in state is included) or object related information onthe random object. In case of receiving the input of the selectionaction, the mobile terminal 100 is able to display path information 930on a path from a current terminal position to the random object underthe control of the controller 180.

This is explained in detail with reference to FIG. 10A and FIG. 10B asfollows.

FIG. 10A and FIG. 10B are diagrams of screen configurations fordisplaying path information on a path from a current terminal positionto a specific object in case of receiving a selection of a specificobject included in an image according to the present invention.

Referring to FIG. 10A, in case of receiving an input of a selectionaction for a specific object 601 [a], the mobile terminal 100 is able tosearch path information on a path from a current terminal position tothe specific object 601 and provide a search status message 1010 [b]. Indoing so, in order to obtain the path information on the path from thecurrent terminal position to the specific object, the mobile terminal100 is able to drive a navigation function.

Referring to FIG. 10B, the mobile terminal 100 is able to display thefound path information on the path from the current terminal position1002 to the specific object 1001 on a map image.

Moreover, the found path information is stored in the memory 160 or canbe used as road guide information according to a user selection.

According to the present invention, after a zoom-in is performed in apredetermined distance from a current terminal position, in case that adirection of the camera 121 is changed, the mobile terminal 100 displaysan image corresponding to the change direction on the assumption that apoint corresponding to the predetermined distance is a terminal positionor is able to display an image corresponding to the changed direction bymaintaining the predetermined distance zoom-in at the current terminalposition, under the control of the controller 180.

This is explained in detail with reference to FIGS. 11A to 11C asfollows.

FIGS. 11A to 11C are diagrams of screen configurations for a case that acamera direction is changed in a zoom-in state according to the presentinvention.

Referring to FIG. 11A, a reference number 1101 indicates a currentterminal position. And, a reference number 1102 indicates a pointcorresponding to a predetermined distance after a predetermined distancezoom-in has been performed. Moreover, in case of a predetermineddistance zoom-in, it is able to assume that a ΔΔ movie theater 1110 islocated in front.

Referring to FIG. 11B and FIG. 11C, assume a case that the camera 121 isrotated by 90 degrees counterclockwise in a predetermined distancezoom-in state. In FIG. 11B and FIG. 11C, assume that the predetermineddistance zoom-in state is maintained despite that a camera direction ischanged.

Referring to FIG. 11B, assuming that a terminal is located at apredetermined distance point 1102, the mobile terminal 100 is able todisplay an image including an object 1120 located at a point rotatedcounterclockwise from the predetermined distance point 1102.

Referring to FIG. 11C, the mobile terminal 100 is able to display animage including an object 1130 located at a point rotatedcounterclockwise from a current terminal position 1101.

In the following description, a second embodiment of the presentinvention is explained with reference to FIGS. 12 to 17.

FIG. 12 is a flowchart of a method of displaying object relatedinformation in a mobile terminal according to a second embodiment of thepresent invention.

Referring to FIG. 12, the mobile terminal 100 receives an input of animage externally via the camera [S1210]. In this case, the input imageis a preview image and can be displayed via the display module 151.

Descriptions of the image input and display can refer to the formerdescription of the inputting step S310 shown in FIG. 3.

And, a process for obtaining terminal position information and objectrelated information can refer to the former description.

The mobile terminal 100 displays the image input in the inputting stepS1210 and also displays the obtained object related information withinthe input image, under the control of the controller 180 [S1220]. Inthis case, the displaying step S1220 can be performed using the displaymodule 151.

Regarding the displaying step S1120, the description of the formerdisplaying step S330 shown in FIG. 3 is referred to.

The mobile terminal 100 externally receives a command signal for anidentifiable object display via the user input unit 130 [S1230]. In thiscase, the identifiable object display can mean that a specific objectcorresponding to specific object related information is identifiablydisplayed within the input image. And, the specific object relatedinformation can include one of at least one or more object relatedinformation corresponding to the input image.

For instance, the command signal for the identifiable object display canbe input if a user selects a corresponding menu item, a correspondingkey or a corresponding key zone.

For instance, referring to FIG. 13A, if a user touches an identifiabledisplay zone 1301 provided to a prescribed region on the screen, themobile terminal 100 can receives the command signal for the identifiableobject display.

Alternatively, the mobile terminal 100 determines a definition of theinput image under the control of the controller 180 [S1241].

In this case, the definition indicates an extent of user's ability inrecognizing an object included in the input image and can depend on atleast a climate (e.g., raining, snowing, cloudy, etc.), a time zone(e.g., night, day, dawn, etc.), or intensity of illumination.

For instance, if the user's recognition ability extent is low, thedefinition is low as well. If the user's recognition ability extent ishigh, the definition is high as well. In particular, it can be said thatthe definition is low in case of rain, snow, night or high intensity ofillumination.

The mobile terminal 100 determines whether the determined definition isequal to or smaller than a predetermined reference under the control ofthe controller 180 [S1243].

In this case, the predetermined reference is set by a user or can be setby the terminal.

For instance, referring to FIG. 13B, if the definition is equal to orsmaller than the predetermined reference, the mobile terminal 100displays a window 1302 for indicating the definition equal to or smallerthan the predetermined reference on the screen. If the user selects an“OK” zone, the mobile terminal 100 is able to perform the identifiableobject display.

Of course, if the definition is equal to or smaller than thepredetermined reference, the mobile terminal 100 is able toautomatically perform the identifiable object display (explained later).

In case of receiving the input of the command signal of the identifiableobject display in the inputting step S1230 or determining that thedefinition is equal to or smaller than the predetermined reference inthe determining step S1243, the mobile terminal 100 determines whetherthe specific object corresponding to the specific object relatedinformation is included in the input image under the control of thecontroller 180 [S1250].

In this case, the controller 180 is able to check the object relatedinformation on the object included in the input image among the at leastone or more object related information corresponding to the input image.

In case of determining that the specific object corresponding to thespecific object related information is included in the determining stepS1250, the mobile terminal 100 identifiably displays the specific objectwithin the input image under the control of the controller 180 [S1260].In this case, the identifiably displaying step S1260 can be performedusing the display module 151.

In the following description, the identifiably displaying step S1260 isexplained in detail with reference to detailed embodiments.

According to a first embodiment, the mobile terminal 100 is able toidentifiably display specific object related information among aplurality of object related information corresponding to the input imageunder the control of the controller 180.

This is explained with reference to FIG. 13C and FIG. 13D as follows.

Referring to FIG. 13C, the mobile terminal 100 is able to identifiablydisplay Doosan Tower 1311 having a corresponding object exist in animage among object related information (i.e., Doosan Tower 1311, XXElectronics 1312 and ΔΔ Hospital 1313) corresponding to the image.

Referring to FIG. 13D, in case that the object related informationhaving corresponding objects exist in the image are the Doosan Tower1311 and the ΔΔ Hospital 1313, the mobile terminal 100 is able toidentifiably display both of them. The mobile terminal 100 is then ableto receive a selection of the object related information correspondingto the object to display identifiably from a user.

Therefore, the mobile terminal 100 is able to identifiably display theobject corresponding to the Doosan Tower 1311 selected by the user only.

Of course, the mobile terminal 100 is able to identifiably display bothof the Doosan Tower 1311 and the ΔΔ Hospital 1313 as well.

According to a second embodiment, under the control of the controller180, the mobile terminal 100 transmits a request signal for a similarimage of the input image to the external server and is then able toreceive the similar image from the external server as a result of thetransmission of the request signal. In this case, the request signaltransmitting operation and the similar image receiving operation can beperformed using the wireless communication unit 110.

In this case, the similar image means an image including at least one ormore objects included in the input image entirely or in part or can meanan image having position information equal to position information(e.g., address information, direction information, etc.) correspondingto the input image. Moreover, the similar image can include an imagecorresponding to current terminal position information (directioninformation included).

The request signal can include current terminal position information,position information of the input image, position information of anobject included in the input image and the like. Therefore, the externalserver searches the similar image using the information included in therequest signal and then transmits the searched similar image to themobile terminal 100.

Alternatively, if a plurality of images is stored in the memory 160 ofthe mobile terminal 100, the mobile terminal 100 is able to search aplurality of the stored images for the similar image. In this case,position information can be designated to each of a plurality of thestored images. Therefore, the controller 180 is able to search for sucha similar image as an image to which position information equal toposition information of the input image is designated.

The mobile terminal 100 extracts a partial image corresponding to thespecific object from the similar image and is then able to display theinput image in a manner of replacing a partial image corresponding tothe specific object in the input image by the extracted partial image,under the control of the controller 180. In doing so, the display of theobject related information on the specific object can be maintained.

Alternatively, the mobile terminal 100 is able to display the similarimage instead of the input image in a manner of replacing the inputimage by the similar image, under the control of the controller 180. Indoing so, the display of the object related information corresponding tothe input image can be maintained.

This is explained with reference to FIGS. 14A to 14C as follows.

FIGS. 14A to 14C are diagrams of screen configurations for displaying aspecific object identifiably using a similar image according to thepresent invention.

Referring to FIG. 14A, in case of searching the memory 160 for a similarimage, the mobile terminal 100 outputs a phrase 1410 indicating that asimilar image search is in progress [a]. In case of receiving a similarimage from an external server, the mobile terminal 100 is able to outputa phrase 1420 indicating that a similar image reception is in progress[b].

Referring to FIG. 14B, the mobile terminal 100 displays the similarimage in a manner of replacing the input image by the similar image andis able to display specific object related information 1311 on aspecific object 1321 within the similar image.

Referring to FIG. 14C, the mobile terminal 100 extracts an image partcorresponding to a specific object 1321 in the similar image and is thenable to display the input image in a manner of replacing an image partcorresponding to the specific object 1321 in the input image by theextracted image part.

According to a third embodiment, the mobile terminal 100 is able toreceive 3D graphic information on a specific object from an externalserver using the wireless communication unit 110 under the control ofthe controller 180. In this case, prior to receiving the 3D graphicinformation on the specific object, the mobile terminal 100 is able totransmit a request signal for the 3D graphic information on the specificobject.

For instance, the request signal can include position information (e.g.,address information, direction information, etc.) of the specificobject. Hence, the external server searches 3D graphic information of anobject corresponding to the position information of the specific objectincluded in the request signal and is then able to transmit the found 3Dgraphic information to the mobile terminal 100.

This is explained with reference to FIGS. 15A to 15C as follows.

FIGS. 15A to 15C are diagrams of screen configurations for displaying aspecific object identifiably using a 3D graphic information according tothe present invention.

Referring to FIG. 15A, in case of receiving 3D graphic information on aspecific object from an external server, the mobile terminal 100 is ableto output a phrase 1510 indicating that a 3D graphic informationreception is in progress.

Referring to FIG. 15B, the mobile terminal 100 is able to display aspecific object 1321 displayed 3 Dimensionally using the received 3Dgraphic information instead of a partial image corresponding to thespecific object in the input image.

Referring to FIG. 15C, in case of receiving a rotation command signalfor a 3D displayed specific object 1321 [a], the mobile terminal 100 isable to display the specific object 1321 in a manner of rotating thespecific object 1321 by a predetermined angle in a directioncorresponding to the rotation command signal [b].

For instance, in case of receiving an input of a touch & drag action onthe specific object 1321 in a predetermined direction or an input of aninclining action in a predetermined direction of the terminal, therotation command signal can be input.

According to a fourth embodiment, the mobile terminal 100 obtainsoutline information of a specific object in the input image and is thenable to display the specific object identifiably using the obtainedoutline information, under the control of the controller 180.

This is explained with reference to FIG. 16A and FIG. 16B as follows.

FIG. 16A and FIG. 16B are diagrams of screen configurations fordisplaying a specific object identifiably using an outline informationaccording to the present invention.

Referring to FIG. 16A, in case of extracting outline information of aspecific object 1321 in the input image, the mobile terminal 100 is ableto output a phrase 1610 indicating that an outline informationextraction is in progress.

Referring to FIG. 16B, the mobile terminal 100 is able to identifiablydisplay the specific object 1321 in a manner of thickening the outlineof the specific object 1321.

According to the present invention, if the definition of the image isequal to or smaller than a predetermined reference or a brightnessadjust command signal for the input image is input by a user, the mobileterminal 100 is able to adjust brightness of the input image under thecontrol of the controller 180.

The mobile terminal 100 adjusts the brightness of the input image,thereby displaying the input image more clearly. This is explained withreference to FIG. 17 as follows.

FIG. 17 is a diagram of screen configuration for controlling a displayof a preview image using brightness adjustment according to the presentinvention.

Referring to FIG. 17, if the definition of the image is equal to orsmaller than a predetermined reference or a brightness adjust commandsignal for the input image is input by a user, the mobile terminal 100is able to display a brightness adjust bar 1710 on a prescribed regionof a screen in the course of a display of the input image. Therefore,the user is able to adjust the brightness of the input image using thebrightness adjust bar 1710.

Accordingly, the present invention provides the following effects and/oradvantages.

First of all, the present invention provides different object relatedinformation on the same object according to a zoom-in extent of animage, thereby obtaining user-specific object related information byadjusting the zoom-in extent.

Secondly, in case of relieving an input of a zoom-in command signal in azoom-in unavailable state, the present invention provides object relatedinformation in accordance with a zoom-in operation, thereby providing auser with more various kinds of information including internal objectinformation related to a specific object, information on an objectblocked by a specific object and the like in the zoom-in unavailablestate.

Thirdly, if definition of an image input via camera is lowered, thepresent invention identifiably displays a specific object included inthe image, thereby informing a user that the corresponding objectrelated information relates to the specific object despite the lowereddefinition of the image.

According to one embodiment of the present invention, theabove-described object related information displaying methods can beimplemented in a program recorded medium as computer-readable codes. Thecomputer-readable media include all kinds of recording devices in whichdata readable by a computer system are stored. The computer-readablemedia include ROM, RAM, CD-ROM, magnetic tapes, floppy discs, opticaldata storage devices, and the like for example and also includecarrier-wave type implementations (e.g., transmission via Internet).And, the computer can include the controller 180 of the terminal.

The aforementioned embodiments are achieved by combination of structuralelements and features of the present invention in a predetermined type.Each of the structural elements or features should be consideredselectively unless specified separately. Each of the structural elementsor features may be carried out without being combined with otherstructural elements or features. Also, some structural elements and/orfeatures may be combined with one another to constitute the embodimentsof the present invention.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present inventionwithout departing from the spirit or scope of the inventions. Thus, itis intended that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents.

1. A mobile terminal comprising: a camera module configured to receivean image that depicts at least one object located within a predetermineddistance from the mobile terminal; a controller configured to obtain atleast one object related information corresponding to the received imageand control the camera module to zoom in on the received image accordingto a zoom-in command signal; and a display module configured to displaythe zoomed-in image at a zoom position and object related information inthe obtained at least one object related information corresponding to atleast one object of the zoomed-in image depicted between the zoomposition and the predetermined distance.
 2. The mobile terminal of claim1, wherein the at least one object related information corresponding tothe received image includes at least information related to the at leastone object depicted in the received image or information related to atleast one object depicted in a direction in which the camera modulefaces and located within the predetermined distance.
 3. The mobileterminal of claim 1, further comprising: a memory configured to storeobject related information for a plurality of objects; and a wirelesscommunication unit configured to receive object related information froman external server, wherein the controller is further configured toeither search the stored object related information or use the receivedobject related information to obtain the at least one object relatedinformation corresponding to the received image.
 4. The mobile terminalof claim 1, wherein the controller is further configured to set theobtained at least one object related information to differ for differentzoom positions of the same zoomed-in image.
 5. The mobile terminal ofclaim 1, wherein the controller is further configured to control thedisplay module to display internal object related informationcorresponding to a specific object included in the zoomed-in image asthe obtained at least one object related information when the zoomed-inimage cannot be zoomed in on any further.
 6. The mobile terminal ofclaim 5, further comprising: a memory configured to store a plurality ofinternal object related information including the internal objectrelated information corresponding to the specific object; and a wirelesscommunication unit configured to receive the internal object relatedinformation.
 7. The mobile terminal of claim 1, wherein the controlleris further configured to control the display module to displayinformation corresponding to at least one object blocked by a specificobject included in the zoomed-in image as the obtained at least oneobject related information when the zoomed-in image cannot be zoomed inon any further.
 8. The mobile terminal of claim 7, wherein the displayedinformation corresponding to the at least one blocked object includes atleast map information, a virtual image, or point information of the atleast one blocked object.
 9. The mobile terminal of claim 1, furthercomprising: a user input unit configured to receive a selection actionfor a specific object included in the received image or object relatedinformation corresponding to the specific object, wherein the controlleris further configured to control the display module to display pathinformation of a path depicted between a particular position of themobile terminal and the specific object when the selection action isreceived.
 10. The mobile terminal of claim 1, wherein the controller isfurther configured to: determine whether object related informationcorresponding to a specific object is present among the obtained atleast one object related information in order to determine if thespecific object is included in the received image, and control thedisplay module to display the specific object including a symbol toidentify the specific object when the specific object is determined tobe included in the received image.
 11. The mobile terminal of claim 10,further comprising: a user input unit configured to receive a commandsignal for an identifiable object display, wherein the controller isfurther configured to control the display module to perform theidentifiable object display by displaying the symbol relative to thespecific object when the command signal is received.
 12. The mobileterminal of claim 10, wherein the controller is further configured to:determine a definition of the received image, and control the displaymodule to display the specific object identifiably when the determineddefinition is less than or equal to a predetermined reference value. 13.The mobile terminal of claim 10, further comprising: a wirelesscommunication unit configured to transmit a request signal to anexternal server for an image similar to the received image and toreceive the requested similar image from the external server in responseto the transmitted request signal, wherein the controller is furtherconfigured to extract a portion of the received similar imagecorresponding to the specific object and control the display module todisplay the extracted portion such that the extracted portion replaces aportion of the received image corresponding to the specific object. 14.The mobile terminal of claim 10, further comprising: a wirelesscommunication unit configured to receive three-dimensional graphicinformation corresponding to the specific object from an externalserver, wherein the controller is further configured to control thedisplay module to display the specific object identifiably using thereceived three-dimensional graphic information.
 15. The mobile terminalof claim 10, wherein the controller is further configured to: obtainoutline information corresponding to the specific object from thereceived image, and control the display module to display the specificobject identifiably using the obtained outline information.
 16. A methodof displaying object related information in a mobile terminal, themethod comprising: receiving an image via a camera module of the mobileterminal, the received image depicting objects located within apredetermined distance from the mobile terminal; obtaining at least oneobject related information corresponding to the received image;displaying the received image including the obtained at least one objectrelated information; receiving a zoom-in command signal for the cameramodule; zooming in on the displayed image according to the receivedzoom-in command signal such that the zoomed-in image is displayed at azoom position; and displaying object related information in the obtainedobject related information corresponding to at least one object of thezoomed-in image depicted between the zoom position and the predetermineddistance.
 17. The method of claim 16, further comprising: storing objectrelated information for a plurality of objects in a memory of the mobileterminal; and receiving the object related information from an externalserver, wherein obtaining the at least one object related informationcomprises searching the stored object related information or using thereceived object related information.
 18. The method of claim 16, whereindisplaying the obtained object related information comprises setting theobtained at least one object related information to differ for differentzoom positions of the same zoomed-in image.
 19. The method of claim 16,wherein displaying the obtained object related information comprisesdisplaying internal object related information corresponding to aspecific object included in the zoomed-in image as the obtained at leastone object related information when the zoomed-in image cannot be zoomedin on any further.
 20. The method of claim 16, wherein displaying theobtained object related information comprises displaying informationcorresponding to at least one object blocked by a specific objectincluded in the zoomed-in image as the obtained at least one objectrelated information when the zoomed-in image cannot be zoomed in on anyfurther.
 21. The method of claim 16, further comprising: determiningwhether object related information corresponding to a specific object ispresent among the obtained at least one object related information inorder to determine if the specific object is included in the receivedimage; and displaying the specific object including a symbol to identifythe specific object when the specific object is determined to beincluded in the received image.